Method and system for generating image sample having specific feature

ABSTRACT

The present application provides a method and a system for generating an image sample having a specific feature. The method includes: training a generative adversarial network-based sample generation model, where the generative adversarial network includes a generator and two discriminators: a global discriminator configured to perform global discrimination on an image, and a local discriminator configured to perform local discrimination on a specific feature; and inputting, to a trained generator that serves as a sample generation model, a semantic segmentation image that indicates a location of the specific feature and a corresponding real image not having the specific feature, to obtain a generated image sample having the specific feature.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International applicationPCT/CN2021/135284 filed on Dec. 3, 2021. This application isincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present application relates to the field of artificial intelligence,and in particular, to a method and a system for generating an imagesample having a specific feature.

BACKGROUND ART

With the development of artificial intelligence technologies, artificialintelligence is increasingly applied to various scenarios. In the fieldof image recognition, many technologies that use artificial intelligenceto improve the accuracy of image recognition have been developed.

In such artificial intelligence-based image recognition technologies,usually, an image recognition model is first trained based on a trainingdata set, and then an image to be recognized is input to a trainedmodel, and the model outputs a recognition result. A training process ofthe model is time-consuming, and a recognition accuracy rate of themodel quite depends on accuracy and an amount of training data. However,it may be difficult to obtain a sufficient amount of accurate trainingdata. For example, to train a model for recognizing a specific featureincluded in an image, a large number of image samples having thespecific feature are required. However, when a natural occurrenceprobability of the specific feature is quite low, it is difficult toacquire a real image having such a specific feature.

SUMMARY OF THE INVENTION

In view of the above problems, the present application provides a methodand a system for generating an image sample having a specific feature,to properly increase a number of samples when there are only a quitesmall number of such samples, so as to quickly acquire a sufficientnumber of required samples.

According to a first aspect, the present application provides a methodfor training a sample generation model for generating an image samplehaving a specific feature. The method includes: obtaining a trainingdata set, where the training data set includes a plurality of realimages having a specific feature, a corresponding semantic segmentationimage, and a plurality of real images not having the specific feature,and the semantic segmentation image is a binary image for distinguishingthe specific feature from another object; constructing a generativeadversarial network, where a generator of the generative adversarialnetwork is configured to generate a generative image having the specificfeature based on the input semantic segmentation image and acorresponding real image not having the specific feature, where thesemantic segmentation image is used as a priori information about alocation of the specific feature; and a discriminator of the generativeadversarial network further includes a global discriminator and a localdiscriminator, where the global discriminator is configured todiscriminate authenticity of an input image, and the local discriminatoris configured to discriminate authenticity of an input local imagehaving the specific feature; and performing adversarial training on thegenerative adversarial network to optimize an ability of the generatorto generate a generative image having the specific feature based on areal image not having the specific feature, where the trained generatoris used as the sample generation model.

Through this method for training a sample generation model, an imagesample generation model capable of generating a specific feature at aspecified location in a real image can be obtained through training.Compared with a conventional generative adversarial network, the presentapplication introduces information about a location of a specificfeature as a priori information. This can effectively reduce an amountof computation during training of the generator and the discriminator ofthe generative adversarial network, and improve a discriminationaccuracy rate and generation fidelity. In addition, an additional localdiscriminator is added, so that a generated feature can remain realisticin local zoom-in, and details of the generated feature can be betterrestored.

In some embodiments, the generator uses a SPADE-based generatorstructure, and the generator is configured to: use, as an input, datathat includes the semantic segmentation image and an image obtainedafter erasing, from a corresponding real image, information about aspecific feature region indicated in the semantic segmentation image,where the image obtained after erasing the information is input to abackbone network of an encoder-decoder structure, and the semanticsegmentation image is input, as a semantic mask, to a SPADE branch to beintroduced into a decoder part of the backbone network. The backbonenetwork uses the image obtained after erasing the information as randomnoise to generate a feature map of the specific feature region indicatedin the semantic segmentation image, performs blending, based on thesemantic segmentation image, on the generated feature map and the imageobtained after erasing the information, and uses a blended image as thegenerated generative image having the specific feature. By using thisSPADE-based generator structure, the generator can generate an image ofa defect at a specified location in the semantic segmentation imagebased on a feature of an input real image of a sealing pin region. Thegenerated defect appears to have a feature of a surrounding region, andblending with the surrounding region is quite natural and realistic. Inaddition, in a final output picture, only the defect region isgenerated, so that a remaining part is completely free of distortion. Inaddition, computation processes can be reduced during encoding anddecoding of the generator. For example, compared with a whole image,only an image of a quite small defect region needs to be generated.

In some embodiments, the performing adversarial training on thegenerative adversarial network includes: separately labeling the realimage and the generative image having the specific feature that isgenerated by the generator, and providing the images together with acorresponding semantic segmentation image to the global discriminatorfor training; and cutting, based on the semantic segmentation image, alocal real image and a local generative image that have a specificfeature part respectively from the labeled real image provided to theglobal discriminator and the generative image having the specificfeature hat is generated by the generator, and providing, to the localdiscriminator for training, the local real image and the localgenerative image together with a local semantic segmentation imagehaving the specific feature part that is cut from the semanticsegmentation image. A local image is cut from a global image for use bythe local discriminator, without increasing a burden of additionaltraining sample data. Likewise, the semantic segmentation image onlyneeds to be cut correspondingly without additional preparation.

In some embodiments, data used for training the discriminator includes:a training data pair that includes a labeled real image having thespecific feature and a corresponding real semantic segmentation image, atraining data pair that includes a labeled real image not having thespecific feature and a semantic segmentation image selected from thereal semantic segmentation image, and a training data pair that includesa labeled real image not having the specific feature and a randomlygenerated semantic segmentation image. In this manner, a samediscriminator algorithm and program can be applied to a picture nothaving a defect and a picture having the defect, and a burden of arequirement for a number of training samples is also reduced.

In some embodiments, the performing adversarial training on thegenerative adversarial network includes: training the globaldiscriminator and the local discriminator in parallel. By integratingglobal and local discrimination results, the discriminator has astronger discrimination ability than a conventional discriminator, sothat a generation effect of the generator is further improved.Particularly, a local image generation ability with respect to a defectlocation is significantly improved.

In some embodiments, the specific feature is a welding defect of asealing pin. The method for training a sample generation model in thepresent application is used to train a model for generating a sealingpin welding defective sample, thereby effectively solving a problem thata number of real defective samples is small.

According to a second aspect, the present application provides a methodfor generating an image sample having a specific feature. The methodincludes: obtaining a real image not having a specific feature;constructing a semantic segmentation image that indicates a location ofa region in which the specific feature is expected to be generated,where the semantic segmentation image is a binary image fordistinguishing the specific feature from another object; erasinginformation, in the real image not having the specific feature, thatcorresponds to the region having the specific feature in the constructedsemantic segmentation image; and inputting an image obtained aftererasing the information, together with the semantic segmentation image,to a sample generation model trained according to a model trainingmethod provided in the present application, to obtain an image samplehaving the specific feature.

Through this method, a defect can be generated, according to arequirement, at a specified location in a real image not having thedefect, to form an image having the defect as a sealing pin weldingdefective sample, so that a large number of image samples available fortraining can be conveniently obtained.

In some embodiments, the constructing a semantic segmentation image thatindicates a location of a region in which the specific feature isexpected to be generated includes: selecting one semantic segmentationimage from a plurality of real semantic segmentation imagescorresponding to a plurality of real images having the specific feature;or specifying one or more expected locations for the specific feature ona plurality of real images not having the specific feature, andgenerating a corresponding semantic segmentation image based on thespecified one or more expected locations for the specific feature. Asemantic segmentation image of a real defective image is directly used,or an expected location of a defect is directly specified, for example,a location on a sealing pin weld bead is specified, so that a generatedimage sample having the defect can be closer to a real defective image.

In some embodiments, the specific feature is a welding defect of asealing pin. The method for generating an image sample having a specificfeature in the present application is used to generate a sealing pinwelding region image having a defect, thereby effectively solving aproblem that a number of real sealing pin welding defective samples issmall.

According to a third aspect, the present application provides a systemfor training a sample generation model for generating an image samplehaving a specific feature. The system includes: a training dataacquisition module configured to obtain a training data set, where thetraining data set includes a plurality of real images having a specificfeature, a corresponding semantic segmentation image, and a plurality ofreal images not having the specific feature, and the semanticsegmentation image is a binary image for distinguishing the specificfeature from another object; and a model training module configured to:construct a generative adversarial network, where a generator of thegenerative adversarial network is configured to generate a generativeimage having the specific feature based on the input semanticsegmentation image and a corresponding real image not having thespecific feature, where the semantic segmentation image is used as apriori information about a location of the specific feature; and adiscriminator of the generative adversarial network further includes aglobal discriminator and a local discriminator, where the globaldiscriminator is configured to discriminate authenticity of an inputimage, and the local discriminator is configured to discriminateauthenticity of an input local image having the specific feature; andperform adversarial training on the generative adversarial network tooptimize an ability of the generator to generate a generative imagehaving the specific feature based on a real image not having thespecific feature, where the trained generator is used as the samplegeneration model. Through the system for training a sample generationmodel for generating an image sample having a specific feature in thepresent application, an image sample generation model capable ofgenerating a specific feature at a specified location in a real imagecan be obtained through training, so that a large number of trainingsamples required can be quickly generated.

In some embodiments, the generator uses a SPADE-based generatorstructure, and the generator is configured to: use, as an input, datathat includes the semantic segmentation image and an image obtainedafter erasing, from a corresponding real image, information about aspecific feature region indicated in the semantic segmentation image,where the image obtained after erasing the information is input to abackbone network of an encoder-decoder structure, and the semanticsegmentation image is input, as a semantic mask, to a SPADE branch to beintroduced into a decoder part of the backbone network. The backbonenetwork uses the image obtained after erasing the information as randomnoise to generate a feature map of the specific feature region indicatedin the semantic segmentation image, performs blending, based on thesemantic segmentation image, on the generated feature map and the imageobtained after erasing the information, and uses a blended image as thegenerated generative image having the specific feature. By using thisSPADE-based generator structure, the generator can generate an image ofa defect at a specified location in the semantic segmentation imagebased on a feature of an input real image of a sealing pin region. Thegenerated defect appears to have a feature of a surrounding region, andblending with the surrounding region is quite natural and realistic. Inaddition, in a final output picture, only the defect region isgenerated, so that a remaining part is completely free of distortion. Inaddition, computation processes can be reduced during encoding anddecoding of the generator. For example, compared with a whole image,only an image of a quite small defect region needs to be generated.

In some embodiments, the performing adversarial training on thegenerative adversarial network includes: separately labeling the realimage and the generative image having the specific feature that isgenerated by the generator, and providing the images together with acorresponding semantic segmentation image to the global discriminatorfor training; and cutting, based on the semantic segmentation image, alocal real image and a local generative image that have a specificfeature part respectively from the labeled real image provided to theglobal discriminator and the generative image having the specificfeature hat is generated by the generator, and providing, to the localdiscriminator for training, the local real image and the localgenerative image together with a local semantic segmentation imagehaving the specific feature part that is cut from the semanticsegmentation image. A local image is cut from a global image for use bythe local discriminator, without increasing a burden of additionaltraining sample data. Likewise, the semantic segmentation image onlyneeds to be cut correspondingly without additional preparation.

In some embodiments, data used for training the discriminator includes:a training data pair that includes a labeled real image having thespecific feature and a corresponding real semantic segmentation image, atraining data pair that includes a labeled real image not having thespecific feature and a semantic segmentation image selected from thereal semantic segmentation image, and a training data pair that includesa labeled real image not having the specific feature and a randomlygenerated semantic segmentation image. In this manner, a samediscriminator algorithm and program can be applied to a picture nothaving a defect and a picture having the defect, and a burden of arequirement for a number of training samples is also reduced.

In some embodiments, the performing adversarial training on thegenerative adversarial network includes: training the globaldiscriminator and the local discriminator in parallel. By integratingglobal and local discrimination results, the discriminator has astronger discrimination ability than a conventional discriminator, sothat a generation effect of the generator is further improved.Particularly, a local image generation ability with respect to a defectlocation is significantly improved.

In some embodiments, the specific feature is a welding defect of asealing pin. The system for training a sample generation model in thepresent application is used to train a model for generating a sealingpin welding defective sample, thereby effectively solving a problem thata number of real defective samples is small.

According to a fourth aspect, the present application provides a systemfor generating an image sample having a specific feature. The systemincludes: an image acquisition module configured to obtain a real imagenot having a specific feature; a semantic segmentation imageconstruction module configured to construct a semantic segmentationimage that indicates a location of a region in which the specificfeature is expected to be generated, where the semantic segmentationimage is a binary image for distinguishing the specific feature fromanother object; an image erasing module configured to erase information,in the real image not having the specific feature, that corresponds tothe region having the specific feature in the constructed semanticsegmentation image; and a sample generation module configured to inputan image obtained after erasing the information, together with thesemantic segmentation image, to a sample generation model trainedaccording to a model training method provided in the presentapplication, to obtain an image sample having the specific feature.Through this system, a defect can be generated, according to arequirement, at a specified location in a real image not having thedefect, to form an image having the defect as a sealing pin weldingdefective sample, so that a large number of image samples available fortraining can be conveniently obtained.

In some embodiments, the constructing a semantic segmentation image thatindicates a location of a region in which the specific feature isexpected to be generated includes: selecting one semantic segmentationimage from a plurality of real semantic segmentation imagescorresponding to a plurality of real images having the specific feature;or specifying one or more expected locations for the specific feature ona plurality of real images not having the specific feature, andgenerating a corresponding semantic segmentation image based on thespecified one or more expected locations for the specific feature. Asemantic segmentation image of a real defective image is directly used,or an expected location of a defect is directly specified, for example,a location on a sealing pin weld bead is specified, so that a generatedimage sample having the defect can be closer to a real defective image.

In some embodiments, the specific feature is a welding defect of asealing pin. The system for generating an image sample having a specificfeature in the present application is used to generate a sealing pinwelding region image having a defect, thereby effectively solving aproblem that a number of real sealing pin welding defective samples issmall.

According to a fifth aspect, a system for generating an image samplehaving a specific feature is provided. The system includes: a storageunit configured to store a real image not having a specific feature, asample generation model trained by using a model training methodprovided in the present application, and a generated image sample havingthe specific feature; and a computing unit configured to: construct asemantic segmentation image that indicates a location of a region inwhich the specific feature is expected to be generated, where thesemantic segmentation image is a binary image for distinguishing thespecific feature from another object; read the real image not having thespecific feature from the storage unit; erase information, in the realimage not having the specific feature, that corresponds to the regionhaving the specific feature in the constructed semantic segmentationimage; and use an image obtained after erasing the information, togetherwith the semantic segmentation image, as an input to the samplegeneration model, to obtain an image sample having the specific feature.

Through the system for generating an image sample having a specificfeature in the present application, a problem that a number of realsamples having the specific feature is small can be effectively solved,and an approximately real training image sample is generated by usingthe trained sample generation model.

The above description is only an overview of the technical solutions ofthe present application. In order to more clearly understand thetechnical means of the present application to implement same accordingto the contents of the specification, and in order to make the above andother objects, features, and advantages of the present application moreobvious and understandable, specific embodiments of the presentapplication are exemplarily described below.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to illustrate the technical solutions of the embodiments of thepresent application more clearly, the drawings required in thedescription of the embodiments of the present application will bedescribed briefly below. Obviously, the drawings described below aremerely some embodiments of the present application, and for those ofordinary skill in the art, other drawings can also be obtained fromthese drawings without any creative efforts.

FIG. 1 is an example flowchart of a method for training a samplegeneration model for generating an image sample having a specificfeature according to an embodiment of the present application;

FIG. 2 is an example of a real image sample not having a specificfeature according to an embodiment of the present application;

FIG. 3 is an example semantic segmentation image corresponding to thereal image sample in FIG. 2 ;

FIG. 4 is an example structural diagram of a generator according to anembodiment of the present application;

FIG. 5 is an example structural diagram of a discriminator according toan embodiment of the present application;

FIG. 6 is an example flowchart of a method for generating an imagesample having a specific feature according to an embodiment of thepresent application;

FIG. 7 is an example structural diagram of a system for training asample generation model for generating an image sample having a specificfeature according to an embodiment of the present invention;

FIG. 8 is an example structural diagram of a system for generating animage sample having a specific feature according to an embodiment of thepresent invention; and

FIG. 9 is an example structural diagram of a system for generating animage sample having a specific feature according to an embodiment of thepresent invention.

In the accompanying drawings, the figures are not drawn to scale.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of the technical solutions of the present application willbe described in more detail below with reference to the accompanyingdrawings. The following embodiments are merely intended to more clearlyillustrate the technical solutions of the present application, so theymerely serve as examples, but are not intended to limit the scope ofprotection of the present application.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meanings as those commonly understood by those skilled inthe art to which the present application belongs. The terms used hereinare merely for the purpose of describing specific embodiments, but arenot intended to limit the present application. The terms “including” and“having” and any variations thereof in the description and the claims ofthe present application as well as the brief description of theaccompanying drawings described above are intended to covernon-exclusive inclusion.

In the description of the embodiments of the present application, thetechnical terms “first”, “second”, etc. are merely used fordistinguishing different objects, and are not to be construed asindicating or implying relative importance or implicitly indicating thenumber, particular order or primary-secondary relationship of thetechnical features modified thereby. In the description of theembodiments of the present application, the phrase “a plurality of”means two or more, unless otherwise explicitly and specifically defined.

The phrase “embodiment” mentioned herein means that the specificfeatures, structures, or characteristics described in conjunction withthe embodiment can be encompassed in at least one embodiment of thepresent application. The phrase at various locations in the descriptiondoes not necessarily refer to the same embodiment, or an independent oralternative embodiment exclusive of another embodiment. Those skilled inthe art understand explicitly or implicitly that the embodimentdescribed herein may be combined with another embodiment.

In the description of the embodiments of the present application, theterm “and/or” is merely intended to describe the associated relationshipof associated objects, indicating that three relationships can exist,for example, A and/or B can include: the three instances of A alone, Aand B simultaneously, and B alone. In addition, the character “/” hereingenerally indicates an “or” relationship between the associated objects.

Sealing pin welding is an essential link in a production process oftraction batteries. Whether the sealing pin welding meets the standarddirectly affects the safety of the batteries. A sealing pin weldingregion is referred to as a weld bead. Due to a change in a temperature,an environment, or the like during welding, defects such as a pinhole, aburst point, a burst line (pseudo soldering), missing welding, and amelted bead usually occur on the weld bead. Whether the weld bead has adefect can be automatically detected based on visual AI. Weld beadlocating is the first step of defect detection. Therefore, it is quitenecessary to develop a precise weld bead locating algorithm.

To train this type of image recognition model, a large number of imagesnot having a defect and images having the defect are required aspositive and negative samples respectively for training. However, due toan existing manufacturing process and production quality control, aprobability of defects in product manufacturing is quite low. Therefore,real defective samples account only for a quite small part of totalproduction, increasing difficulty of model training. In addition, it isalso time-consuming to collect a sufficient number of real defectivesamples.

In view of the above, to solve a problem that a number of trainingsamples, especially a number of negative samples, is quite small, thepresent application provides a sample increase method. Morespecifically, a generative adversarial network may be constructed andtrained to obtain a sample generation model capable of generating agenerative image having a defect based on an input real image not havingthe defect. By using this model, a number of negative samples can beeffectively increased, and a training time for an image recognitionmodel and a preparation time for system go-live can be shortened.

It may be understood that the present application may be widely appliedto the field of quality detection combined with artificial intelligence.A sample generation method and system disclosed in the embodiments ofthe present application may be but is not limited to being used togenerate a sealing pin welding defective sample for a traction battery,and may be further used to generate a defective sample for any othertype of product in modern industrial manufacturing or generate any imagesample having a specific feature.

In the following embodiments, for the convenience of description,generating a sealing pin welding defective sample for a traction batteryis taken as an example for description.

According to an embodiment of the present application, FIG. 1 is anexample flowchart of a method 100 for training a sample generation modelfor generating an image sample having a specific feature according to anembodiment of the present application. As shown in FIG. 1 , the method100 starts at step 101 in which a training data set is obtained. Thetraining data set includes a plurality of real images having a specificfeature, a corresponding semantic segmentation image, and a plurality ofreal images not having the specific feature. The semantic segmentationimage is a binary image for distinguishing the specific feature fromanother object. In step 102, a generative adversarial network isconstructed. A generator of the generative adversarial network isconfigured to generate a generative image having the specific featurebased on the input semantic segmentation image and a corresponding realimage not having the specific feature, where the semantic segmentationimage is used as a priori information about a location of the specificfeature. A discriminator of the generative adversarial network furtherincludes a global discriminator and a local discriminator, where theglobal discriminator is configured to discriminate authenticity of aninput image, and the local discriminator is configured to discriminateauthenticity of an input local image having the specific feature. Instep 103, adversarial training is performed on the generativeadversarial network to optimize an ability of the generator to generatea generative image having the specific feature based on a real image nothaving the specific feature. The trained generator is used as the samplegeneration model.

The real image is a real image captured by an image capturing device(for example, a camera). For example, in this example, the real imagemay be a photo of a sealing pin welding region, as shown in FIG. 2 . Thespecific feature refers to an object that is included in an image andthat has a specific feature, for example, the sky, the sea, a cat, or adog. In this example, the specific feature may be a defect in thesealing pin welding region.

FIG. 3 is a semantic segmentation image corresponding to the real imagein FIG. 2 . The semantic segmentation image is also referred to as amask image. In this example, the semantic segmentation image identifiesonly the specific feature in the image, namely, the defect in thesealing pin welding region. Therefore, in FIG. 3 , a white regionrepresents a defect part, and a remaining part represents a non-defectpart. Therefore, the semantic segmentation image provides informationabout whether the specific feature exists in the image and a specificlocation of the specific feature.

The generative adversarial network is an existing unsupervised machinelearning model in the prior art. A typical generative adversarialnetwork includes a generative network and a discriminative network,which are also referred to as a generator and a discriminator. In agenerative adversarial network model used to generate an image, agenerator is trained to generate an expected picture, and adiscriminator is trained to determine whether an input picture is a realpicture or a picture generated by the generator. If a discriminationsuccess rate of the discriminator is quite high, it indicates thatcurrent generation fidelity of the generator is quite low, and thereforea generative algorithm of the generator needs to be optimized.Otherwise, if a discrimination success rate of the discriminator isquite low, it indicates that a current discriminative algorithm of thediscriminator needs to be optimized. As the discriminator continuouslyoptimizes its discriminative algorithm and the generator continuouslyoptimizes its generative algorithm, the discriminator and the generatorfinally reach a balance. In this case, a discrimination success rate ofthe discriminator is close to 50%, that is, the discriminator cannotdiscriminate authenticity of a picture, in other words, a picturegenerated by the generator is infinitely close to a real picture.

In this example, an objective is to obtain, through training, a modelcapable of generating a generative image having a defect based on aninput real image not having the defect and a semantic segmentation image(indicating a defect location). Therefore, the generator continuouslygenerates a generative image having a defect by using a real image nothaving the defect and a corresponding semantic segmentation image, forthe discriminator to perform discrimination. In a generation process, adefective image is used as a priori information indicating a defectlocation, so that the defect in the generative image is generated at adefect location indicated in the semantic segmentation image.

Different from the prior art, the discriminator in the presentapplication further includes a global discriminator and a localdiscriminator. The global discriminator has a same operating principleas that of a discriminator in the prior art, and is configured todiscriminate authenticity of a complete input image. The localdiscriminator is configured to discriminate authenticity of an imagethat has a defect part and that is cut from the complete input image. Adiscriminative algorithm of the local discriminator may be the same asthat of the global discriminator, and the local discriminator and theglobal discriminator may use a convolutional neural network CNN with asame structure.

As mentioned above, an overall training process for the generativeadversarial network is a process of “adversarial” training of thegenerator and the discriminator, until the generator and thediscriminator reach a balance. In this example, the adversarial trainingis similar to a common training process for the generative adversarialnetwork, except that a discrimination result provided by an originaldiscriminator is currently obtained by integrating respectivediscrimination results of the global discriminator and the localdiscriminator. A final training result is that neither the globaldiscriminator nor the local discriminator can distinguish authenticityof an image having the defect that is generated by the generator.Therefore, the trained generator may be used to generate a defectiveimage at a specified location based on a real image not having thedefect, so as to obtain a generative image having the defect. Thegenerator may also be used as a sample generation model for generatingan image sample having the sealing pin welding defect.

Through this method for training a sample generation model provided inthe present application, an image sample generation model capable ofgenerating a specific feature at a specified location in a real imagecan be obtained through training. Compared with a conventionalgenerative adversarial network, the sample generation model in thepresent application introduces information about a location of aspecific feature as a priori information. This can effectively reduce anamount of computation during training of the generator and thediscriminator of the generative adversarial network, and improve adiscrimination accuracy rate and generation fidelity. In addition, anadditional local discriminator is added, so that a generated feature canremain realistic in local zoom-in, and details of the generated featurecan be better restored.

According to some embodiments of the present application, optionally,the generator uses a SPADE-based generator structure, and the generatoris configured to: use, as an input, data that includes the semanticsegmentation image and an image obtained after erasing, from acorresponding real image, information about a specific feature regionindicated in the semantic segmentation image, where the image obtainedafter erasing the information is input to a backbone network of anencoder-decoder structure, and the semantic segmentation image is input,as a semantic mask, to a SPADE branch to be introduced into a decoderpart of the backbone network. The backbone network uses the imageobtained after erasing the information as random noise to generate afeature map of the specific feature region indicated in the semanticsegmentation image, performs blending, based on the semanticsegmentation image, on the generated feature map and the image obtainedafter erasing the information, and uses a blended image as the generatedgenerative image having the specific feature.

FIG. 4 is an example structural diagram of a generator according to anembodiment of the present application. As shown in FIG. 4 , a backbonenetwork of the generator uses a common encoder-decoder structure. Aninput to the encoder is an image obtained after information about aspecific feature region indicated in a semantic segmentation image iserased from an original image. In this example, the original image is asealing pin welding region, and the semantic segmentation image is amask image indicating a location of a welding defect. Therefore, theimage input to the encoder is an image obtained after a welding defectregion indicated in the mask image is erased. In an optional example, askip link may be added between the generator and the encoder.

SPADE is an existing algorithm in the field of image generation, and canobtain a variety of composed images when a semantic segmentation imageis given. A typical application is to learn a style of a reference imageand transform a style of an input image based on a semantic segmentationimage, for example, change a landscape photo into an oil painting style.In this example, the SPADE (Spatially-Adaptive Normalization) algorithmis introduced into a decoding part of the generator as a branch networkof the generator, to assist the decoder in completing a normalizationoperation. The semantic segmentation image indicating the defectlocation is provided to the SPADE as an input, and the defect locationis also introduced into a generation model as a priori information.

An output of the encoder is a generative image obtained after a regionis erased, that is, a generated defective image. Blending is performedon the generative image and the input image of the generator (that is,the image obtained after the defect region is erased) based on thelocation indicated in the mask image, to obtain a generated sealing pinwelding image having the defect.

By using this SPADE-based generator structure, the generator cangenerate an image of a defect at a specified location in the semanticsegmentation image based on a feature of an input real image of asealing pin region. The generated defect appears to have a feature of asurrounding region, and blending with the surrounding region is quitenatural and realistic. In addition, in a final output picture, only thedefect region is generated, so that a remaining part is completely freeof distortion. In addition, computation processes can be reduced duringencoding and decoding of the generator. For example, compared with awhole image, only an image of a quite small defect region needs to begenerated.

According to some embodiments of the present application, optionally,the performing adversarial training on the generative adversarialnetwork includes: separately labeling the real image and the generativeimage having the specific feature that is generated by the generator,and providing the images together with a corresponding semanticsegmentation image to the global discriminator for training; andcutting, based on the semantic segmentation image, a local real imageand a local generative image that have a specific feature partrespectively from the labeled real image provided to the globaldiscriminator and the generative image having the specific feature hatis generated by the generator, and providing, to the local discriminatorfor training, the local real image and the local generative imagetogether with a local semantic segmentation image having the specificfeature part that is cut from the semantic segmentation image.

FIG. 5 is an example structural diagram of a discriminator according toan embodiment of the present application. As mentioned above, the localdiscriminator and the global discriminator in the present applicationmay be based on a same neural network structure, and a difference liesonly in that the local discriminator performs discriminative training byusing a local image cut from a whole image (hereinafter referred to as a“global image”). Alternatively, the local discriminator may be trainedby using a local image sample set different from a training sample setused for the global discriminator. During training of the globaldiscriminator, a labeled global image and a corresponding semanticsegmentation image may be used to perform training. A real picture,regardless of a real picture having a defect or a real image not havinga defect, may be labeled as “true”, and a generative picture may belabeled as “false”, to verify discrimination accuracy of thediscriminator. Similarly, during training of the local discriminator, alocal image and a local semantic segmentation image that have a defectpart, for example, a rectangular local image having a defect part, maybe cut from a global image and a semantic segmentation image, andlabeling may still be used. In a discrimination process of thediscriminator, a semantic segmentation image corresponding to an imagemay be used as a priori indication indicating a defect location, thatis, the discriminator knows the defect location, and may discriminateauthenticity by focusing on analysis of data at the defect location.

A local image is cut from a global image for use by the localdiscriminator, without increasing a burden of additional training sampledata. Likewise, the semantic segmentation image only needs to be cutcorrespondingly without additional preparation.

According to some embodiments of the present application, optionally,data used for training the discriminator includes: a training data pairthat includes a labeled real image having the specific feature and acorresponding real semantic segmentation image, a training data pairthat includes a labeled real image not having the specific feature and asemantic segmentation image selected from the real semantic segmentationimage, and a training data pair that includes a labeled real image nothaving the specific feature and a randomly generated semanticsegmentation image.

The training data for the discriminator is used to enable thediscriminator to learn of a feature of a real picture and a feature of agenerative picture, so as to distinguish between the two pictures. Areal semantic segmentation image corresponding to a real picture nothaving a defect should be empty. However, this does not meet a trainingrequirement, because the discriminator needs to be notified of specificlocations on which discrimination is to be focused, and specific partsthat are to be cut and provided to the local discriminator, similar to acase of a picture having a defect. Therefore, in a training process, areal picture not having a defect may be paired with a semanticsegmentation image of a real picture having the defect, or may be pairedwith a randomly generated semantic segmentation image. Optionally, areal picture not having a defect may be alternatively paired withsemantic segmentation images of a plurality of different real pictureshaving the defect and a randomly generated semantic segmentation imageto form a plurality of sets of training data. This reduces a number ofreal pictures not having a defect that need to be used as trainingsamples.

In this manner, a same discriminator algorithm and program can beapplied to a picture not having a defect and a picture having thedefect, and a burden of a requirement for a number of training samplesis also reduced.

According to some embodiments of the present application, optionally,the performing adversarial training on the generative adversarialnetwork includes: training the global discriminator and the localdiscriminator in parallel.

As mentioned above, the local discriminator may perform discriminativetraining by using a local image and a local semantic segmentation imagethat are cut from a global image and a semantic segmentation image thatare used by the global discriminator. Therefore, global training dataand local training data are respectively provided to the twodiscriminators, so that the two discriminators can be trained inparallel, thereby further reducing an overall time spent in adiscriminator training process.

According to some embodiments of the present application, optionally,the performing adversarial training on the generative adversarialnetwork includes: comprehensively evaluating a generation effect of thegenerator in combination with discrimination results of the globaldiscriminator and the local discriminator, and optimizing a parameter ofthe generator based on a comprehensive evaluation result of thediscriminators.

As mentioned above, the discrimination results of the globaldiscriminator and the local discriminator may be integrated as adiscrimination result. For example, the result is “true” or “false” onlywhen the discrimination results of the two discriminators are the same,for example, when both discrimination results are “true” (that is, areal image) or both discrimination results are “false” (that is, agenerative image); and the result is “false” when the discriminationresults of the two discriminators are different. The result is thencompared with a label of input training data to verify a discriminationaccuracy rate of the discriminator. The discrimination accuracy rate ofthe discriminator being closer to 50% indicates a better generationeffect of the generator. Similar to a conventional generativeadversarial network that uses only one discriminator, the results of thediscriminator may be fed back to the generator for parameteroptimization, to improve the generation effect of the generator.

By integrating global and local discrimination results, thediscriminator has a stronger discrimination ability than a conventionaldiscriminator, so that a generation effect of the generator is furtherimproved. Particularly, a local image generation ability with respect toa defect location is significantly improved.

According to some embodiments of the present application, optionally,the specific feature is a welding defect of a sealing pin.

As mentioned above, the method for training a sample generation model inthe present application may be used to train a model for generating asealing pin welding defective sample, thereby effectively solving aproblem that a number of real defective samples is small.

According to another embodiment of the present application, FIG. 6 is anexample flowchart of a method 600 for generating an image sample havinga specific feature according to an embodiment of the presentapplication. As shown in FIG. 6 , the method 600 starts at step 601 inwhich a real image not having a specific feature is obtained. In step602, a semantic segmentation image that indicates a location of a regionin which the specific feature is expected to be generated isconstructed, where the semantic segmentation image is a binary image fordistinguishing the specific feature from another object. In step 603,information, in the real image not having the specific feature, thatcorresponds to the region having the specific feature in the constructedsemantic segmentation image is erased. In step 604, an image obtainedafter erasing the information is input, together with the semanticsegmentation image, to a sample generation model trained according to amethod for training a sample generation model in the presentapplication, to obtain an image sample having the specific feature.

A real image may be obtained by using an image capturing device, forexample, obtained through photographing by using a camera. For eachobtained real image, a location at which the specific feature isexpected to be generated may be determined. In an example of an image ofa sealing pin welding region, a location at which a defect is expectedto be generated, for example, a specific location on a weld bead, may bedetermined. Each image may have more than one defect location, that is,may have a plurality of defect locations. The location may be manuallyspecified, or may be randomly specified on the weld bead after alocation of the weld bead in the image is obtained through imagerecognition and analysis. After the location is determined, a binarysemantic segmentation image is constructed correspondingly. For example,the defect is labeled, and a non-defect part is not labeled. Thencontent at a corresponding location in the real image is erased based onthe location indicated in the constructed semantic segmentation image.An image obtained after the erasing is paired with the semanticsegmentation image as an input to the sample generation model. Thesample generation model is the aforementioned model that is trained togenerate, based on a semantic segmentation image and an image obtainedafter erasing content at a corresponding location, an image having aspecific feature at a corresponding location. Therefore, an image samplehaving a defect at a specified location in the input real image isobtained as an output of the model.

Through this method, a defect can be generated, according to arequirement, at a specified location in a real image not having thedefect, to form an image having the defect as a sealing pin weldingdefective sample, so that a large number of image samples available fortraining can be conveniently obtained.

According to some embodiments of the present application, optionally,the constructing a semantic segmentation image that indicates a locationof a region in which the specific feature is expected to be generatedincludes: selecting one semantic segmentation image from a plurality ofreal semantic segmentation images corresponding to a plurality of realimages having the specific feature; or specifying one or more expectedlocations for the specific feature on a plurality of real images nothaving the specific feature, and generating a corresponding semanticsegmentation image based on the specified one or more expected locationsfor the specific feature.

A semantic segmentation image of a real defective image is directlyused, or an expected location of a defect is directly specified, forexample, a location on a sealing pin weld bead is specified, so that agenerated image sample having the defect can be closer to a realdefective image.

According to some embodiments of the present application, optionally,the specific feature is a welding defect of a sealing pin.

As mentioned above, the method for generating an image sample having aspecific feature in the present application may be used to generate asealing pin welding region image having a defect, thereby effectivelysolving a problem that a number of real sealing pin welding defectivesamples is small.

According to still another embodiment of the present application,referring to FIG. 7 , the present application provides a system 700 fortraining a sample generation model for generating an image sample havinga specific feature. As shown in FIG. 7 , the system 700 includes: atraining data acquisition module 701 and a model training module 702.The training data acquisition module 701 is configured to obtain atraining data set, where the training data set includes a plurality ofreal images having a specific feature, a corresponding semanticsegmentation image, and a plurality of real images not having thespecific feature, and the semantic segmentation image is a binary imagefor distinguishing the specific feature from another object. The modeltraining module 702 is configured to construct a generative adversarialnetwork, where a generator of the generative adversarial network isconfigured to generate a generative image having the specific featurebased on the input semantic segmentation image and a corresponding realimage not having the specific feature, where the semantic segmentationimage is used as a priori information about a location of the specificfeature. A discriminator of the generative adversarial network furtherincludes a global discriminator and a local discriminator. The globaldiscriminator is configured to discriminate authenticity of an inputimage, and the local discriminator is configured to discriminateauthenticity of an input local image having the specific feature. Themodel training module 702 is further configured to perform adversarialtraining on the generative adversarial network to optimize an ability ofthe generator to generate a generative image having the specific featurebased on a real image not having the specific feature. The trainedgenerator is used as the sample generation model.

According to some embodiments of the present application, optionally,the generator uses a SPADE-based generator structure, and the generatoris configured to: use, as an input, data that includes the semanticsegmentation image and an image obtained after erasing, from acorresponding real image, information about a specific feature regionindicated in the semantic segmentation image, where the image obtainedafter erasing the information is input to a backbone network of anencoder-decoder structure, and the semantic segmentation image is input,as a semantic mask, to a SPADE branch to be introduced into a decoderpart of the backbone network. The backbone network uses the imageobtained after erasing the information as random noise to generate afeature map of the specific feature region indicated in the semanticsegmentation image, performs blending, based on the semanticsegmentation image, on the generated feature map and the image obtainedafter erasing the information, and uses a blended image as the generatedgenerative image having the specific feature.

According to some embodiments of the present application, optionally,the performing adversarial training on the generative adversarialnetwork includes: separately labeling the real image and the generativeimage having the specific feature that is generated by the generator,and providing the images together with a corresponding semanticsegmentation image to the global discriminator for training; andcutting, based on the semantic segmentation image, a local real imageand a local generative image that have a specific feature partrespectively from the labeled real image provided to the globaldiscriminator and the generative image having the specific feature hatis generated by the generator, and providing, to the local discriminatorfor training, the local real image and the local generative imagetogether with a local semantic segmentation image having the specificfeature part that is cut from the semantic segmentation image.

According to some embodiments of the present application, optionally,data used for training the discriminator includes: a training data pairthat includes a labeled real image having the specific feature and acorresponding real semantic segmentation image; a training data pairthat includes a labeled real image not having the specific feature and asemantic segmentation image selected from the real semantic segmentationimage; and a training data pair that includes a labeled real image nothaving the specific feature and a randomly generated semanticsegmentation image.

According to some embodiments of the present application, optionally,the performing adversarial training on the generative adversarialnetwork includes: training the global discriminator and the localdiscriminator in parallel.

According to some embodiments of the present application, optionally,the performing adversarial training on the generative adversarialnetwork includes: comprehensively evaluating a generation effect of thegenerator in combination with discrimination results of the globaldiscriminator and the local discriminator, and optimizing a parameter ofthe generator based on a comprehensive evaluation result of thediscriminators.

According to some embodiments of the present application, optionally,the specific feature is a welding defect of a sealing pin.

Through the system for training a sample generation model for generatingan image sample having a specific feature in the present application, animage sample generation model capable of generating a specific featureat a specified location in a real image can be obtained throughtraining, so that a large number of training samples required can bequickly generated. This model training system is configured to performthe model training method in the present application, and therefore alsocorrespondingly has the technical effects of the embodiments of themodel training method. For brevity, details are not described hereinagain.

According to yet another embodiment of the present application,referring to FIG. 8 , the present application provides a system 800 forgenerating an image sample having a specific feature. As shown in FIG. 8, the system 800 includes: an image acquisition module 801, the imageacquisition module 801 being configured to obtain a real image nothaving a specific feature; a semantic segmentation image constructionmodule 802, the semantic segmentation image construction module 802being configured to construct a semantic segmentation image thatindicates a location of a region in which the specific feature isexpected to be generated, where the semantic segmentation image is abinary image for distinguishing the specific feature from anotherobject; an image erasing module 803, the image erasing module 803 beingconfigured to erase information, in the real image not having thespecific feature, that corresponds to the region having the specificfeature in the constructed semantic segmentation image; and a samplegeneration module 804, the sample generation module 804 being configuredto input an image obtained after erasing the information, together withthe semantic segmentation image, to a sample generation model trainedaccording to a model training method provided in the presentapplication, to obtain an image sample having the specific feature.

According to some embodiments of the present application, optionally,the constructing a semantic segmentation image that indicates a locationof a region in which the specific feature is expected to be generatedincludes: selecting one semantic segmentation image from a plurality ofreal semantic segmentation images corresponding to a plurality of realimages, of welding of a sealing pin, that have the specific feature; orspecifying one or more expected locations for the specific feature on aplurality of real images not having the specific feature, and generatinga corresponding semantic segmentation image based on the specified oneor more expected locations for the specific feature.

According to some embodiments of the present application, optionally,the specific feature is a welding defect of a sealing pin.

Corresponding to the sample generation method in the presentapplication, the system for generating an image sample having a specificfeature in the present application may be used to generate a sealing pinwelding region image having a defect, thereby effectively solving aproblem that a number of real sealing pin welding defective samples issmall.

According to still yet another embodiment of the present application,referring to FIG. 9 , a system 900 for generating an image sample havinga specific feature is provided. As shown in FIG. 9 , the system 900includes: a storage unit 901 configured to store a real image not havinga specific feature, a sample generation model trained by using a modeltraining method provided in the present application, and a generatedimage sample having the specific feature; and a computing unit 902, thecomputing unit 902 being configured to: construct a semanticsegmentation image that indicates a location of a region in which thespecific feature is expected to be generated, where the semanticsegmentation image is a binary image for distinguishing the specificfeature from another object; read the real image not having the specificfeature from the storage unit 901; erase information, in the real imagenot having the specific feature, that corresponds to the region havingthe specific feature in the constructed semantic segmentation image; anduse an image obtained after erasing the information, together with thesemantic segmentation image, as an input to the sample generation model,to obtain an image sample having the specific feature.

The storage unit 901 may include a RAM, a ROM, or a combination thereof.In some cases, the storage unit 901 may include, in particular, a basicinput/output system (BIOS) that may control basic hardware or softwareoperations, such as interaction with peripheral components or devices.The stored real image may be an image sample obtained by using an imagecapturing device.

The computing unit 902 may include an intelligent hardware device (forexample, a general-purpose processor, a digital signal processor (DSP),a central processing unit (CPU), a microcontroller, anapplication-specific integrated circuit (ASIC), a field programmablegate array (FPGA), a programmable logic device, a discrete gate ortransistor logic component, a discrete hardware component, or anycombination thereof).

The various illustrative blocks and modules described in connection withthe disclosure herein can be implemented or performed with ageneral-purpose processor, a DSP, an ASIC, an FPGA, or anotherprogrammable logic device, discrete gate, or transistor logic, adiscrete hardware component, or any combination thereof, that isdesigned to perform functions described herein. The general-purposeprocessor may be a microprocessor, but in an alternative, the processormay be any conventional processor, controller, microcontroller, or statemachine. The processor may be alternatively implemented as a combinationof computing devices (for example, a combination of a DSP and amicroprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfigurations). The functions described herein may be implemented inhardware, software executed by a processor, firmware, or any combinationthereof. If implemented in software executed by a processor, thefunctions may be stored on or transmitted over a computer-readablemedium as one or more instructions or codes. Other examples andimplementations are within the scope of the present disclosure and theappended claims. For example, due to the nature of software, thefunctions described herein may be implemented by using software executedby a processor, hardware, firmware, hardwiring, or any combinationthereof. Features implementing the functions may also be physicallylocated at various locations, including being distributed such thatportions of the functions are implemented at different physicallocations.

Through the system for generating an image sample having a specificfeature in the present application, a problem that a number of realsamples having the specific feature is small can be effectively solved,and an approximately real training image sample is generated by usingthe trained sample generation model. This system may be used to generatean image sample having a defect of a sealing pin welding region, and mayalso be widely applied to various scenarios in which a large number ofimage samples having a specific feature need to be generated.

While the present application has been described with reference to thepreferred embodiments, various modifications can be made, andequivalents can be provided to substitute for the components thereofwithout departing from the scope of the present application. Inparticular, the technical features mentioned in the embodiments can becombined in any manner, provided that there is no structural conflict.The present application is not limited to the specific embodimentsdisclosed herein but includes all the technical solutions that fallwithin the scope of the claims.

The invention claimed is:
 1. A method for training a sample generationmodel for generating an image sample having a specific feature, whereinthe method comprises: obtaining a training data set, wherein thetraining data set comprises a plurality of real images having a specificfeature, a corresponding semantic segmentation image, and a plurality ofreal images not having the specific feature, and the semanticsegmentation image is a binary image for distinguishing the specificfeature from another object; constructing a generative adversarialnetwork, wherein a generator of the generative adversarial network isconfigured to generate a generative image having the specific featurebased on the input semantic segmentation image and a corresponding realimage not having the specific feature, wherein the semantic segmentationimage is used as a priori information about a location of the specificfeature; and a discriminator of the generative adversarial networkfurther comprises a global discriminator and a local discriminator,wherein the global discriminator is configured to discriminateauthenticity of an input image, and the local discriminator isconfigured to discriminate authenticity of an input local image havingthe specific feature; and performing adversarial training on thegenerative adversarial network to optimize an ability of the generatorto generate a generative image having the specific feature based on areal image not having the specific feature, wherein the trainedgenerator is used as the sample generation model.
 2. The method of claim1, wherein the generator uses a SPADE-based generator structure, and thegenerator is configured to: use, as an input, data that comprises thesemantic segmentation image and an image obtained after erasing, from acorresponding real image, information about a specific feature regionindicated in the semantic segmentation image, wherein the image obtainedafter erasing the information is input to a backbone network of anencoder-decoder structure, and the semantic segmentation image is input,as a semantic mask, to a SPADE branch to be introduced into a decoderpart of the backbone network, wherein the backbone network uses theimage obtained after erasing the information as random noise to generatea feature map of the specific feature region indicated in the semanticsegmentation image; and performs blending, based on the semanticsegmentation image, on the generated feature map and the image obtainedafter erasing the information, and uses a blended image as the generatedgenerative image having the specific feature.
 3. The method of claim 1,wherein the performing adversarial training on the generativeadversarial network comprises: separately labeling the real image andthe generative image having the specific feature that is generated bythe generator, and providing the images together with a correspondingsemantic segmentation image to the global discriminator for training;and cutting, based on the semantic segmentation image, a local realimage and a local generative image that have a specific feature partrespectively from the labeled real image provided to the globaldiscriminator and the generative image having the specific feature hatis generated by the generator, and providing, to the local discriminatorfor training, the local real image and the local generative imagetogether with a local semantic segmentation image having the specificfeature part that is cut from the semantic segmentation image.
 4. Themethod of claim 1, wherein data used for training the discriminatorcomprises: a training data pair that comprises a labeled real imagehaving the specific feature and a corresponding real semanticsegmentation image; a training data pair that comprises a labeled realimage not having the specific feature and a semantic segmentation imageselected from the real semantic segmentation image; and a training datapair that comprises a labeled real image not having the specific featureand a randomly generated semantic segmentation image.
 5. The method ofclaim 1, wherein the performing adversarial training on the generativeadversarial network comprises: training the global discriminator and thelocal discriminator in parallel.
 6. The method of claim 1, wherein theperforming adversarial training on the generative adversarial networkcomprises: comprehensively evaluating a generation effect of thegenerator in combination with discrimination results of the globaldiscriminator and the local discriminator, and optimizing a parameter ofthe generator based on a comprehensive evaluation result of thediscriminators.
 7. The method of claim 1, wherein the specific featureis a welding defect of a sealing pin.
 8. A method for generating animage sample having a specific feature, wherein the method comprises:obtaining a real image not having a specific feature; constructing asemantic segmentation image that indicates a location of a region inwhich the specific feature is expected to be generated, wherein thesemantic segmentation image is a binary image for distinguishing thespecific feature from another object; erasing information, in the realimage not having the specific feature, that corresponds to the regionhaving the specific feature in the constructed semantic segmentationimage; and inputting an image obtained after erasing the information,together with the semantic segmentation image, to a sample generationmodel trained by using a method according to claim 1, to obtain an imagesample having the specific feature.
 9. The method of claim 8, whereinthe constructing a semantic segmentation image that indicates a locationof a region in which the specific feature is expected to be generatedcomprises: selecting one semantic segmentation image from a plurality ofreal semantic segmentation images corresponding to a plurality of realimages having the specific feature; or specifying one or more expectedlocations for the specific feature on a plurality of real images nothaving the specific feature, and generating a corresponding semanticsegmentation image based on the specified one or more expected locationsfor the specific feature.
 10. The method of claim 8, wherein thespecific feature is a welding defect of a sealing pin.
 11. A system forgenerating an image sample having a specific feature, wherein the systemcomprises: a storage unit configured to store a real image not having aspecific feature, a sample generation model trained by using a methodaccording to claim 1, and a generated image sample having the specificfeature; and a computing unit configured to: construct a semanticsegmentation image that indicates a location of a region in which thespecific feature is expected to be generated, wherein the semanticsegmentation image is a binary image for distinguishing the specificfeature from another object; read the real image not having the specificfeature from the storage unit; erase information, in the real image nothaving the specific feature, that corresponds to the region having thespecific feature in the constructed semantic segmentation image; and usean image obtained after erasing the information, together with thesemantic segmentation image, as an input to the sample generation model,to obtain an image sample having the specific feature.
 12. The system ofclaim 11, wherein the constructing a semantic segmentation image thatindicates a location of a region in which the specific feature isexpected to be generated comprises: selecting one semantic segmentationimage from a plurality of real semantic segmentation imagescorresponding to a plurality of real images, of welding of a sealingpin, that have the specific feature; or specifying one or more expectedlocations for the specific feature on a plurality of real images nothaving the specific feature, and generating a corresponding semanticsegmentation image based on the specified one or more expected locationsfor the specific feature.
 13. The system of claim 11, wherein thespecific feature is a welding defect of the sealing pin.